The article discusses the credibility of using Random Forest Variable Importance for identifying causal links in data where the output is binary. It contrasts this method with fitting a Logistic Regression model and examining its coefficients. The discussion highlights the challenges of extracting causality from observational data without controlled experiments, emphasizing the importance of domain knowledge and the use of partial dependence plots for interpreting model results.
In this post, I’ll discuss random forests, another popular approach for feature ranking.
Random forest feature importance
Random forests are among the most popular machine learning methods thanks to their relatively good accuracy, robustness and ease of use. They also provide two straightforward methods for feature selection: mean decrease impurity and mean decrease accuracy.
Random forest feature importance
Random forests are among the most popular machine learning methods thanks to their relatively good accuracy, robustness and ease of use. They also provide two straightforward methods for feature selection: mean decrease impurity and mean decrease accuracy.
Variable importance measures for random forests have been receiving increased attention as a means of variable selection in many classification tasks in bioinformatics and related scientific fields, for instance to select a subset of genetic markers relevant for the prediction of a certain disease. We show that random forest variable importance measures are a sensible means for variable selection in many applications, but are not reliable in situations where potential predictor variables vary in their scale of measurement or their number of categories. This is particularly important in genomics and computational biology, where predictors often include variables of different types, for example when predictors include both sequence data and continuous variables such as folding energy, or when amino acid sequence data show different numbers of categories.
